On the Neural Substrates Leading to the Emergence of Mental 

Operational Structures 

y £>3 

II. 

Department of Electrical Engineering; 

University of Houston 

\VJ Houston, Texas, 77204-4793 



Ogmen 


N 9 3 


z 


e. 




4 /* 


ABSTRACT 

A developmental approach to the study of the emergence of mental operational structures 
in neural networks is presented. Neural architectures proposed to underlie the six stages of the 
sensory-motor period are discussed. 

1 Introduction 

Historically, the study of intelligence has been polarized into explanations based on neurophys- 
iology and those based on logic. The same dichotomy manifests itself in current approaches, 
t lie former corresponding to the neural network theory and the latter to the symbolic reasoning 
schemes used in artificial intelligence (logic, fuzzy logic, etc.). At a first glance these two explana- 
tions fail to extend to each other, for logic does not tell us anything about neurophysiology and it 
seems difficult to explain the rules of logic from the connectivity and firing patterns of neurons. 
However, logic is housed in neurophysiological substrates and there should be a reconciliation 
between these two explanations (if we reject dualism). Since logical reasoning emerges as a result 
of an extensive development and since the early phases of development consist of simpler behav- 
iors. the link between adult intelligence and the underlying neural correlates can be established 
by relating the early developmental stages to neurophysiological substrates and by studying the 
adaptive dynamics of the system that leads to the emergence of higher mental operations. The 
theory of genetic psychology (Piaget. 1907) provides us with a very detailed study of various 
phases starting from birth to the adulthood. This paper extends various psychological concepts 
of the theory of genetic psychology to the neurophysiological domain. In particular, it outlines 
neural networks proposed to give rise to the various stages of the sensory- motor period. 


2 A neural theory of development 

Our study of development is closely linked to the theory of genetic psychology (Piaget, 1907). 
Piaget (1903) named the post-natal developmental period where language is absent the jfiiwri/- 
motor jx riwl and suggested the existence of six consecutive stages that govern its dynamics. 
These stages start from reflexes and end with mental manipulations of sensory-motor schemes to 
invent new intelligent structures. Mental internalization of early sensory-motor schemes lead to 
t he capability of applying them to formal reasoning in the adult life. 
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2.1 The six stages of the sensory-motor level according to the theory of genetic 
psychology 

The first stage of the sensory- motor period consists of simple reflexive sensory-motor behavior. 
In the second stage, the repetitive use of these reflexes, called “primary circular reactions", leads 
to the formation of habits. The primary circular reactions refine genetically encoded reflexes and 
enable the emergence of multi-modal coordination. At the third stage the infant starts to draw 
a distinction between the “means" and “ends" and uses “secondary circular reactions". During 
the fourth stage, existing sensory-motor schemes are coordinated and extended to new situations. 
In stage 5, new sensory-motor schemes are acquired through physical groping. Finally during 
the sixth stage, which marks the end of the sensory-motor level, new sensory-motor schemes are 

acquired by mental groping. 


2.2 A model for primary sensory-motor schemes 

We first start bv describing the seed sensory- motor circuit with nonassociative learning piopei l ies 
( Oginen 1991. 6gmen k Moussa, in press). The architecture, which is illustrated in Figure 1. has 
three main parts: the sensory circuit, the sensory-motor gate circuit, and the motor circuit. Since 
the model was originally formulated explicitly for the prototypical landing behavior of the fly. I lie 
sensory and motor parts are specialized for this animal. The sensory part consists of visual signals 
conveyed bv the compound eves and of tactile pathways. In the fly. signals from the compound 
eves are processed bv three optic ganglia: lamina, medulla, and lobula complex, denoted by La. 
Me. Lo respectively in Figure 1. The output of the visual processing stage results from a bebavioi 
sensitive pooling of motion detector neuron activities. In the case of landing, stimuli indicating 
the approach of a landing site, such as expanding patterns, are detected by an appropriate pooling 
of directionally selective large field motion sensitive neurons and constitute the “agonist input 
to the sensorv-motor gate circuit. This input is denoted by Y a „ in Figure 1. Stimuli of opposite 
character (such as contracting stripes) constitute the “antagonist" input denoted by V Agonist 
and antagonist tactile inputs are added to these visual signals. One such input is marked by 
in Figure 1. The following stage, which is a gated dipole anatomy (Grossberg. 19i2). constitutes 
the sensory-motor gate network because this is the stage where various sensory signals are pooled 
to determine whether a motor command signal will be issued. The sensory-motor gate network 
has agonist and antagonist outputs denoted respectively by .r ag and a„„. These signa s piojec t to 
motor circuits (not shown) to control agonist-antagonist muscle pairs (indicated by Ay *nd A»). 
This sensory-motor model exhibits nonassociative learning as observed in the landing reaction ol 

the llv as well as in human infants (Lipsitt 1990). 

Figure 2 illustrates this architecture augmented with adaptive capabilities. The adaptive ver- 
sion of this sensorv-motor model has also three major parts: sensory, sensory-motor gate, am 
motor. These parts are augmented by inclusion of adaptive mechanisms similar to those proposed 
in the INFANT (Kuperstein k Rubinstein 1989) and AYTTE ( Gaudiano k Grossberg 9- ) mo< - 
els r, represents an environmental variable. These environmental variables are converted into 
neural activities bv the sensory loci. The activities of the sensory loci are denoted by r,. he 
sensorv loci project to the sensory-motor gate networks. The first layer of the sensory motor gate 
network consists of nodes interconnected by recurrent on-center ofl-surround connections. U< i 
node corresponds to the agonist-antagonist inputs of a given sensory- motor scheme The gated 
dipole of Figure 1 is represented in a condensed way by a single node in tins layer An aib.tran 
number of sensorv-motor schemes, similar to the one shown in Figure 1, exists lhe competition 
between these nodes selects and triggers one of the sensory-motor schemes, lhe second layer of 
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Figure 2: Adaptive version of the sensory- mot or model shown in Figure 1 
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the sensory-motor gate part is is used convert the representation of the sensory space into the rep- 
resentations of I lie motor space. The connections converging to this layer are adaptive. The motor 
part consists of three layers and has AV1TE model’s anatomy (Gaudiano k Grossberg 1991 ). The 
output layer produces motor command signals while the input layer receives the desired motor 
command signals. The layer between these “target" and “present" motor signals computes the 
error. This organization constitutes a basic feedback control system where the output is driven 
by an error signal during real-time operation. Moreover, the same "motor error” layer is used to 
adapt to changes in the plant, making the system an adaptive feedback control system. What is 
notable in this anatomy is that the same error signals are used both for performance and learning. 
Another important feature of the AVITE circuit is the “Endogenous Random Generator" (ERG) 
w hich generates random postures and enables the spontaneous learning of sensory-motor coordi- 
nate transforms. The filled circles depict in a condensed wav the function of the ERG. During 
t he active phase of the ERG a random motor signal is dictated to the output of the motor circuit . 
During the passive phase of the ERG, activity of the motor layer is transferred to the first layer 
of the motor circuit which is transferred to the buffer layer of the sensory-motor gate network. 
The arrow from the bottom filled circle to the motor output layer depicts the generation of ran- 
dom motor signals. The pathway from the motor signal level to the motor target level depicts 
the transfer of activity between these layers during the learning phase. A similar transfer occurs 
between the motor target layer and the sensory-motor gate buffer layer. Note that, in addition to 
internal feedback, the sensory-motor circuit constitutes a closed loop through the environmental 
variable denoted by ,r f . The interaction between the environmental variables and neural variables 
is essential and constitutes an overall organization by the relationship of assimilation that unites 
them (Piaget. 1903). 

1 he circuit described above constitutes the proposed basic neural correlate for the reflexive 
behavior of Stage 1. The adaptive nature of the circuit requires the use of these reflexes for 
consolidation ami fine tuning. The repetitive use of reflexes leads to the second stage, the stage of 
habits. Figure 3 shows how the beginnings of “cortexification” occurs at t his stage. In Figure 3. t lie 
architect lire of 1 igure 2 is depicted in a simplified form as a closed loop of environmental, sensory, 
sensory-motor gate, and motor variables denoted by .r t ..r 3 . and .r,„ respectively. To this basic loop, 
additional circuits, proposed to be of cortical origin, are added. These cortical networks receive 
sensorial inputs and have a feedback structure. They are proposed to be adaptive resonance t lieory 
(ART ) architectures (Carpenter A' Grossberg 1983. Grossberg 1976) and each node represents a 
layer of ART. At this stage, sensorial stimuli start to be recorded and generalized rorfically. 
The outputs of these ART circuits also make connections with the sensory-motor circuits. The 
connections from ART circuits to sensory-motor loops are proposed to be Hebbinn synapses so 
that an association between the cortical representation of sensory stimuli and the active sensory- 
motor circuits occurs through the reinforcement of the synaptic weight between the ART circuits 
and the sensory-motor gate nodes. A second feature of cortical development at this stage is the 
beginnings of corl ico-cortico associations. This is shown by synaptic connections between various 
AIM’ circuits. This phase of operation corresponds to Stage 2. in that the sensory-motor schemes 
are not decomposed and consequently there is no differentiation between the "means" and the 
“ends". The lack of decomposition comes from the property that one cannot, at this point, access 
these closed loop circuits from an arbitrary entry point and the only way to activate them is to 
provide the appropriate environmental signals. 

During Stage 3. such a decomposition is introduced by the development of "secondary circular 
reactions". Once (he sensory-motor repertoire becomes sufficiently rich and multi-modal coordi- 
nation (in particular the coordination between vision and grasping) reaches a satisfactory level. 
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Figure 4: Secondary circular reactions 

i he infant starts grasping objects placed in her vicinity. Assume that the infant is presented with 
n toy. grasps the toy. shakes the toy. and that the toy produces a characteristic sound new to the 
infant. This sound will retrigger the sensory-motor scheme, i.e. will generate a circular reaction, 
hater, when the infant is presented with a similar toy. she will try the same sensory- umi or scheme. 
Similarly, when the sound of the toy is reproduced by the experimenter, she will look lor the toy 
(I’iaget. 1963). 

Figured illustrates how this occurs in the proposed neural circuits. Initially, a toy is presented 
( x, i ) and it triggers a sensory- motor scheme (grasping and shaking) whose output is a surprising 
sound ( c, 2 )• The sensory- motor scheme is repeated many times until the novelty of stimuli 
vanishes. 1 During this circular reaction, a connection from the cortical circuits to the sub-cortical 
circuits is also reinforced. If there are consistent stimuli pairings, association between t hese stimuli 
also occurs. In the example of toy shaking, at least an association between the object (r,| ) and 
the sound ( r.j) occurs. Later, assume that the infant is presented with a different object r. . and 
that this object does not- trigger directly the grasping scheme (the dotted connection in figure I). 
Now. if this object has an equivalent cortical generalization, it will send a signal to this seusory- 

1 In the basic architecture of Figure habituating sensory atul sensory- motor gale signals underlie t he habit nat ion 
properties of the circuit. 
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motor scheme via the cortical connection which was reinforced during the circular reactions. If 
this signal is strong enough to trigger the sensory-motor scheme without the sensory signal. I lie 
infant will shake this new object. If this object also makes a similar sound (* e j )■ ' vil1 re activate 
this sensory-motor scheme via a circular reaction. The repetition of the sensory-motor scheme 
will reinforce the direct sub-cortical connection as shown at the bottom of Figure 4. 

While t he previous example considered a single sensory-motor scheme for simplicity, in pract ice 
a larger number of environmental variables (the first being the object, the last being the sound) 

are involved. . 

Note that one can see the important distinction that Piaget drew between his notion of adap- 
tation through an equilibrium between assimilation and accommodation and the associalionistic 
theory of intelligence. While, the behavioral analysis of the network described above may look like 
an associalionistic paradigm, it. is important to emphasize that the associations aie not passiu 
but involve existing sensory-motor schemes. Initially. x (l and the intermediate environmental 
variables are assimilated to generate this particular sensory-motor scheme. During this assimi- 
lation process, other environmental variables are also registered. This registration is primed >v 
the neural activities occurring in the sensory-motor scheme that is active. The time scales or as- 
sociation (i.e. the inter-stimulus intervals) are not arbitrary but are determined by the temporal 
characteristics of the active sensory-motor scheme. Later, when .r <3 is represented to the system, 
it can activate the same sensory-motor scheme either because it is an equivalent stimulus or be- 
cause it was associated with during the previous repetitions. Consider the classical example 
of food bell pairings. The presentation of an object that resembles food will trigger a complex 
sensory-motor scheme in a dog. The same sensory-motor scheme can also be generated by a 
different food. Another possibility- is that a conditioned stimulus, such as the sound ol a bell, .s 
delivered In a fut ure trial, if this bell is delivered alone it will trigger the sensory- motor scheme. 
However, following the first phase (salivation) if the appropriate environmental variables cannot 
be assimilated, the chain of actions will be broken and the global sensory- mot or scheme will be 
aborted (the dogs do not continue to chew etc..). If such a pairing ceases to occur, it will be 
extinguished. However, assume now that the conditioned stimulus not only generates the initial 
stages of t he sensory-motor scheme but also is accompanied by the appropriate intermediary en- 
vironmental variables so that a complete assimilation cycle can occur. The successful completion 
of t he assimilation will reinforce the direct path so that this stimulus will be able to generate the 
whole cycle directly. This change, which consists of an adaptation to the environment, is called 

ticromwodalion (Piaget. 1963). 

Once the sensorv-niotor schemes become accessible via cortical circuits as outlined above, they 
become decomposed and the essential property of Stage 3 emerges: the distinction between means 
and ends. A sensory-motor scheme is the means to achieve a goal. The goal corresponds to I he 
final environmental variable or internal variables associated with this external variable. Assume 
that the infant acquired a sensory-motor scheme consisting of opening a drawer and receiving a 
new tov Initially, this sensory- motor scheme can be generated only by the sight of the drawer. 
Once it becomes decomposed, the infant will look for a drawer with the goal of receiving a new 

0> In stage 4. the decomposed sensory- motor schemes are coordinated into new wholes to achieve 
new goals'. The activation of a new goal can generate multiple or a chain of existing sensory- mot or 
schemes In the case of multiple activation, the competition at the sensory-motor gate level selects 
the best alternative. In the case of a chain, the chain gets activated as a whole as long as the 
appropriate environmental variables can be assimilated. This new grouping becomes reinforced if 
it leads to the achievement of the goal and ran be accessed more readily in the future. 
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Figure '>: Mental groping: Motor layers are actively inhibited and 
sensory-motor schemes without motor reactions. 


i he infant intentionally activates 


If the existing sensory- motor schemes fail to combine with the environmental variables in order 
to complete the cycle leading to the achievement of the goal, new means have to be invented. This 
occurs in Stage ■"> by use of physical groping. Assume that the goal activates some existing sensory- 
motor schemes through the competition at the sensory-tuotor gate layer. This compel it ion will 
start with the best possibility and the system will try different schemes, [fall fails, the F H(i will 
generate random motor behavior or will bias existing sensory- motor schemes. This const it tiles 
the basis of physical groping. If by chance the resulting behavior reaches the goal, it will be 
reinforced through circular reactions and will be integrated to the sensory-motor repertoire as a 
newly discovered means, 

I lie functional operation of Stage 6 is illustrated in Figure 0. At this stage, rhe motor layers 
are actively inhibited. This way. the discovery of new means can be disconnected from actual 
phvsical action. As a result, the subject can carry out "mental groping" and discover new means 
without physical contact. 


3 Concluding remarks 

The early developmental period outlined above indicate that passive perception is inadequate for 
explaining intelligence and the system should actively explore the environment m generate a rich 
repertoire of sensory- motor schemes whose abstractions lead to the formal reasoning structures 
in adult life. 
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